Nearest Neighbor Search using Kd-trees

We suggest a simple modification to the kd-tree search algorithm for nearest neighbor search resulting in an improved performance. The Kd-tree data structure seems to work well in finding nearest neighbors in low dimensions but its performance degrades even if the number of dimensions increases to more than three. Since the exact nearest neighbor search problem suffers from the curse of dimensionality we focus on approximate solutions; a c-approximate nearest neighbor is any neighbor within distance at most c times the distance to the nearest neighbor. We show that for a randomly constructed database of points the traditional kd-tree search algorithm has a very low probability of finding an approximate nearest neighbor; the probability of success drops exponentially in the number of dimensions d as e. However, a simple change to the search algorithm results in a much higher chance of success. Instead of searching for the query point in the kd-tree the search for a random set of points in the neighborhood of the query point. It turns out that searching for e such points can find the c-approximate nearest neighbor with a much higher chance of success.